United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 

WWW.lLSplO.gOV 



APPLICATION NO. 


FILING DATE 


! FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 



10/758,817 



01/16/2004 



Richard L. Marks 



25920 7590 09/27/2007 

MARTINE PEN1LLA & GENCARELLA, LLP 
710 LAKEWAY DRIVE 
SUITE 200 

SUNNYVALE, CA 94085 



SON YP03 1 /SCEA030 1 OUS0O 



6584 



EXAMINER 



WANG, KENT F 



ART UNIT 



PAPER NUMBER 



2622 



MAIL DATE 



09/27/2007 



DELIVERY MODE 



PAPER 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



Office Action Summary 


Application No. 

10/758,817 


Applicant(s) 

MARKS, RICHARD L. 


Examiner 

Im /\ V4 1 I III I W 1 

Kent Wang 


Art Unit 

2622 





The MAILING DATE of this communication appears on the cover sheet with the correspondence address 



Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 133). 
Any reply received by the Office later than three months after t he mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)E3 Responsive to communication(s) filed on 16 July 2007 . 
2a)l3 This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) [3 Claim(s) 1-32 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) El Claim(s) 1-32 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 !)□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



Response to Amendment 

1. The amendments, filed on 07/16/2007, have been entered and made of record. Claims 1-32 
are pending. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1-32 have been considered but are moot in 
view of the new ground(s) or rejection. 



Claim Rejections - 35 USC§ 102 

3. The text of those sections of Title 35, U.S. Code not included in this action can be found in a 
prior Office action. 

4. Claims 1 and 3-32 are rejected under 35 U.S.C. § 102(b) as being anticipated by Gvili, 
"Depth Keying" SPIE Vol. 5006 (2003). 

Regarding claim 1, Gvili discloses a method for differentiating between foreground 
objects and background objects within a scene being captured through an image capture 
device (a novel depth video camera), comprising: 

- emitting a ray of light from a light source toward an object of the scene 
(generating a light wall, Para 3. 1 , pp 566-567); 
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- opening an aperture cover allowing access to a sensor of the image capture 
device for reflected light from the light source (deploying a fast image shutter 
in front of the CCD chip, page 567, lines 3-4); 

- closing the aperture cover after a set time (the shutter is precisely controlling 
the exposure time of the CCD, page 567, lines 17-18), the predefined amount 
of time corresponding to a maximum distance traveled by the light (a real- 
time trimap is generated for each frame based on the original depth matte, 
page 569, lines 22-23); 

- generating a depth mask identifying objects within a foreground region of the 
scene (foreground objects can be generated by setting the depth measurement 
window, page 568, section 3.2) based upon the light captured during the set 
time (a real-time trimap is generated for each frame based on original depth 
matte, page 569, 4th paragraph); and 

- adjusting image capture device parameters according to bit values of the depth 
mask prior to capturing a subsequent corresponding image of the scene 
(normalized depth of pixel D(ij) can be calculated, page 567, lines 7-11) 
wherein the image capture device parameters are selected from a group, 
consisting of focus (zoomed, page 568 line 2), brightness (collected light by 
each pixel, page 567, line 5), exposure (exposure time, page 567, line 18), and 
gain (normalized depth of pixel, page 567, line 8). 

Regarding claim 3, Gvili discloses the light source is configured to emit infrared light 
(IR laser diodes, page 567, lines 15-16). 
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Regarding claim 4, Gvili discloses a method operation of opening an aperture cover 
allowing access to a sensor of the image capture device (the shutter is precisely 
controlling the exposure time of the CCD, page 567, lines 17-18) includes, receiving 
reflected light from the objects within the foreground region (light reflected from every 
object inside the depth measurement window, page 568, second paragraph of section 3.2). 

Regarding claim 5, Gvili discloses a method operation of generating a depth mask 
identifying objects within a foreground region of the scene based upon the light captured 
during the predefined time (depth mask generated according to the depth of the pixel, 
page 569, second paragraph) includes, identifying objects within the foreground region 
with a first bit value (full value pixel representing foreground pixels in the color channel); 
and identifying objects within a background region with a second bit value (zero valued 
pixel representing background) (page 564, section 1). 

Regarding claim 6, Gvili discloses a method operation of adjusting image capture 
device parameters according to bit values of the depth mask prior to capturing a 
subsequent corresponding image of the scene (ability to change the parameters of the 
depth window according to the scenario's need, page 568, section 3.2, first paragraph) 
includes, determining an optimal amount of light based upon the depth mask; and 
adjusting the aperture cover to allow the optimal amount of light into the image capture 
device (normalized depth of pixel can be calculated, page 567, lines 7-11). 

Regarding claim 7, Gvili discloses the image capture device parameters are adjusted 
through mechanical adjustments (depth key setting can be automatically set, page 568, 
third paragraph of section 3.2). 



Application/Control Number: 10/758,817 Page 5 

Art Unit: 2622 

Regarding claim 8, this claim recites same limitations as claim 2. Thus it is analyzed 
and rejected as previously discussed with respect to claim 2 above. 

Regarding claim 9, Gvili discloses a method for adjusting image capture settings for a 
single image capture device (a novel depth video camera), comprising: 

- identifying a scene (easily identified, page 569, third paragraph); 

- capturing an image of the scene through the single image capture device (the 
depth information is captured by a camera, page 566, fifth paragraph); 

- generating a depth mask of the scene from data defining the image of the 
scene (generating depth map according to some depth criterion, page 568, last 
paragraph); and 

- adjusting pixel values of the data defining the image corresponding to objects 
within any one or both of a foreground region and a background region of the 
captured image (normalized depth of pixel D(ij) can be calculated, page 567, 
lines 7-11). 

Regarding claim 10, Gvili discloses the method operation of generating a depth mask 
of the scene from data defining the image of the scene includes segmenting the 
foreground and background regions of the scene (automatic segmentation of the scene is 
possible, page 568, third paragraph of section 3.2). 

Regarding claim 11, Gvili discloses the data defining the image of the scene includes 
pixel data where each pixel is tagged with distance information (Calculate color distances 
between neighboring pixels, page 570, section d). 



Application/Control Number: 1 0/758,81 7 Page 6 

Art Unit: 2622 

Regarding claims 12 and 16, these claims recite same limitations as claim 5. Thus 
they are analyzed and rejected as previously discussed with respect to claim 5 above. 

Regarding claim 13, Gvili discloses the image capture device is selected from the 
group consisting of a digital camera, a web cam, and a camcorder (a novel depth video 
camera, page 564, line 4). 

Regarding claim 14, Gvili discloses the displaying a portion of the image of the scene 
having adjusted pixel values (a new matte is generated by combing the information from 
both the depth and the color frames, page 569, fourth paragraph and also page 571, first 
paragraph of section 4). 

Regarding claim 15, Gvili discloses the portion of the image of the scene is an image 
of a participant for use in an interactive gaming application (man-machine interactions, 
page 572, second paragraph of section 5). 

Regarding claim 17, this claim differs from claim 9 only in that the claim 9 is a 
method claim whereas claim 17 is an apparatus. Thus the apparatus claim 17 is analyzed 
and rejected as previously discussed with respected to claim 9 above. 

Regarding claims 18 and 19, these claims recite same limitations as claims 5 and 4, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 5 and 4 above. 

Regarding claim 20, Gvili discloses each logic element is one or a combination of 
hardware (i.e. shooting lens, depth sensor, as well as a camera) and software (i.e. a depth 
map, depth measurement window, and scene segmentation) (pp. 568-569, section 3.2 
depth key setting) 
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Regarding claim 21, Gvili discloses the image capture device is a video capture 
device (a novel depth video camera, page 564, line 4). 

Regarding claim 22, Gvili discloses the depth logic is further configured to 
periodically provide a depth mask for a sequence of video frames captured by the video 
capture device (step f of the process for iteration of alpha values, page 570). 

Regarding claim 23, Gvili discloses the image capture device setting is adjusted 
through one of a mechanical or electrical adjustment (depth key setting can be 
automatically set, page 568, third paragraph of section 3.2). 

Regarding claim 24, 27, 28, and 31, these claims recite same limitations as claims 16, 
22, 23, and 15, respectively. Thus they are analyzed and rejected as previously discussed 
with respect to claims 16, 22, 23, and 15 above. 

Regarding claim 25, this claim differs from claim 9 only in that the limitations "a 
computing device" and "a display device" are additionally recited. Gvili teaches a 
computing device (a Pentium 3 machine, page 572, section 5) and a display (a video 
camera, page 564, line 4). 

Regarding claim 26, Gvili discloses the computing device is a game console (game 
input device, page 572, section 5). 

Regarding claim 29, Gvili discloses the video capture device is a webcam (ZCam is 
used to generate depth keying during live broadcasts, page 568, line 5 and page 572, 
section 5). 

Regarding claims 30 and 32, these claims recite same limitations as claim 1 1 . Thus 
they are analyzed and rejected as previously discussed with respect to claim 1 1 above. 
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Claim Rejections - 35 USC §103 

5. Claim 2 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Gvili in view of 
Tuomi, US 7,061,507. 

Regarding claim 2, note the discussion of claim 1 above. Gvili does not teach storing the 
depth mask in memory of the image capture device. However, Tuomi teaches storing the 
depth mask in memory of the image capture device (provided a Z-buffer 1902 for storing the 
Z-values relating to the depth of the pixel; see col. 12, lines, 15-36, Tuomi). 

It would have been obvious to one of ordinary skill in the art at the time this invention 
was made to have used a memory as taught by Tuomi as modified by Gvili so that it provide 
multiple buffers for storing information (col. 12, lines 15-17, Tuomi). 

Conclusion 

6. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until 
after the end of the THREE-MONTH shortened statutory period, then the shortened statutory 
period will expire on the date the advisory action is mailed, and any extension fee pursuant to 
37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
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however, will the statutory period for reply expire later than SIX MONTHS from the date of 
this final action. 

7. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Kent Wang whose telephone number is 571-270-1703. The examiner 
can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ngoc-Yen Vu can be reached on 571-272-7320. The fax phone number for the 
organization where this application or proceeding is assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access 
to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 
(toll-free). If you would like assistance from a USPTO Customer Service Representative or 
access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 
571-272-1000. 



KW 

12 September 2007 
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